Added support for restic --ignore-inode #75
Conversation
|
Hi @weisdd Thank you for the PR. Please sign your commit. You can follow the steps recommended by DCO bot: https://github.com/stashed/apimachinery/pull/75/checks?check_run_id=1588113497 Btw, since |
Signed-off-by: Igor Beliakov <[email protected]>
weisdd
force-pushed
the
feature/ignore-inode
branch
from
079d2ba
to
e7025ff
Compare
|
Hi @hossainemruz Thanks for the extra details! |
No. These changes are fine. You will also need to add this here: https://github.com/weisdd/apimachinery/blob/e7025ffbf8c9029da77c75c2e209ce9bad018baa/pkg/restic/commands.go#L165 We have to decide how should we resolve the value of |
|
@hossainemruz I intentionally skipped the function for stdin. My train of thought was that inode becomes an issue only when you mount an external filesystem in userspace and backup files from there. When it's something read from stdin, there should not be an intermediate inode to care about. Plus, the |
|
You are right @weisdd. We don't need |
|
|
|
Any plans to merge the PR or implement the functionality differently? |
|
We are really sorry that it is taking too long. I am currently occupied with a different task. I will return back to Stash in a week or two. Then, I can revisit this PR. |
|
@hossainemruz Honestly, I feel like closing the PR as it's been already 8 months for a simple flag. |
|
Ahh. I forgot about this PR. Sorry, for the delay. I will take a look soon. |
|
@weisdd We have decided to introduce a generic option to pass user arguments to the restic backup/restore command. Check this PR: #121 Now, you can pass any flags (including |
|
@hossainemruz that's even better than my solution, thanks! |
|
Sorry, for this feature request took too long. We were busy with other stuff. Closing this PR now. |
Issue
When backing up data, restic considers the following attributes to decide whether files have changed since the last snapshot:
The last one, inode number, should not be considered in case of FUSE-based filesystems (e.g. mounted via s3-fuse like minio) as they do not support inode consistency, which basically means that inode numbers change on every remount. Otherwise, we'll get two side effects:
=> might slow down the process;
e.g.
"files_new":109,"files_changed":6202,"files_unmodified":525- here, in my test, actually, 109 files were added, but none of 6202 really changed, and 525 happened to have the same inode numbers - just pure luck)=> metrics are also distorted as a consequence.
Note: fortunately, all of that doesn't lead to anything bad like data corruption, it's just some extra work and incorrect statistics.
Proposed change
We only need to add an option to ignore inode changes (
--ignore-inode).Important notes
I'm not an expert in golang nor in your package, so the way I implemented the feature might not be the best one, and maybe there are extra cases where the
--ignore-inodeflag should be called.I tested the change with
v0.10.0-alpha.0for backing up Minio via s3-fuse, and I'm pretty sure it will work with any other version of the package as there's nothing version specific.As for the stash plugins, something simple like this will bring the support wherever it's needed:
cmd.Flags().BoolVar(&opt.setupOptions.IgnoreInode, "ignore-inode", opt.setupOptions.IgnoreInode, "Specify whether restic should ignore inodes (useful for FUSE mounts)")